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DETAILED ACTION 

1 . This action is in regards to the response received on 03/1 0/201 1 . 
Claim 42 has been amended. Claims 42-77 are pending. 



Response to Arguments 

2. The Final Office Action mailed on 01/07/201 1 is vacated and a new 
Non-Final Office Action is set forth herein since Applicant's arguments, see 
remarks, filed 03/10/201 1 , with respect to the rejection of claims 42-59 under 
35 U.S.C. § 103 have been fully considered and are persuasive. Therefore, the 
rejection has been withdrawn. However, upon further consideration, a new 
ground(s) of rejection is made in view of Miner (US 7,653,702 B2). See below. 

3. Applicant's arguments filed 03/10/201 1 with respect to claims 42-77 
have been considered but are moot in view of the new ground(s) of rejection. 



Claim Rejections - 35 USC §102 

The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or 
(2) a patent granted on an application for patent by another filed in the United States before 
the invention by the applicant for patent, except that an international application filed under 
the treaty defined in section 351 (a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United 
States and was published under Article 21 (2) of such treaty in the English language. 
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Claims 42-77 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Miner (US 7,653,702 B2). 

4. With respect to tine claim 42, Miner reference teaclies A metliod for 
retrieving information from a network-based information-provider comprising: 

a) providing a network-enabled, data-capture device configured to retrieve 
information from a network having at least one information provider operative to 
provide search results to queries defined at least in part by electronic 
representations of real-world entities {See column 3, lines 59-67 and see column 
4, lines 1-7, a personal assistant device, and a contextual input device. In use, 
the contextual multimedia association module accesses the Internet (a network- 
enabled j and downloads web documents ("retrieve information from a network; to 
a metadata repository. . . The user captures input data about an object or item of 
interest by means of the contextual input device. . .. The personal assistant device 
automatically digitizes ^electronic representation j and processes the input 
data.... The contextual multimedia association applies the query to numerous 
data stores, optimizes the search results, and then presents the optimized search 
results to the user.) 

b) formulating a query for information to be supplied to a user bv 
transforming data obtained from at least one real-world entity into an electronic 
representation of the real-world entity by way of said data capture device (See 
column 4, lines 1-7, The personal assistant device automatically digitizes 
(transforming data to an electronic representation; and processes the input data, 
and further automatically formulates a query. . . The contextual multimedia 
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association applies the query to numerous data stores, optimizes the search 
results, and then presents the optimized search results to the user); 

c) retrieving information from said network- based, information- 
provider based on tine query {See column 6, lines 1-5, explores the World Wide 
Web by retrieving a document); and 

d) presenting the information by way of a user-output interface {See 
column 8, lines 53-63, displays the preliminary results to the user by means of an 
appropriate output device 330, such as a liquid crystal display (LCD), an audio 
message, a speaker, a monitor, or any other suitable user interface.). 

5. With respect to the claim 43, IVIiner further teaches wherein said data- 
capture device includes an image capture device {See column 10, lines 42-49, a 
camera, that captures the desired image.). 

6. With respect to the claim 44, Miner further teaches wherein said data- 
capture device includes a microphone {See column 1 1, lines 17-29, uses a 
directed microphone as the audio sensor 505.). 

7. With respect to the claim 45, Miner further teaches wherein said data- 
capture device includes a radio receiver (See column 7, lines 20-30, cellular 
radio network 40). 

8. With respect to the claim 46, Miner further teaches wherein said data- 
capture device includes a data-capture device selected from the group consisting 
of scent detector, taste sensor, geophone, motion sensor, acceleration meter, 
wind meter, thermometer, humidly sensor, texture sensor, location sensor, and 
global positioning system receiver (See column 3, lines 47-57, various sensors 
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such as temperature, pressure, humidity, light level, sound, and acceleration 
sensors,). 

9. With respect to tine claim 47, Miner furtlier teaclies wlierein said data- 
capture device is integrated into a cellular phone (See column 7, lines 20-30, 
cellular radio network 40). 

1 0. With respect to the claim 48, Miner further teaches wherein said 
data-capture device is integrated into a device selected from the group consisting 
of a wireless phone, netphone, personal digital assistant, portable computer, 
pager, and personal computer {See column 7, lines 20-30, cellular radio network 
40). 

1 1 . With respect to the claim 49, Miner further teaches wherein said 
network-based information- provider is implemented as a network-based 
dedicated server configured to perform data-processing on data of the electronic 
representation of real-world entities by said data-capture device (See column 4, 
lines 10-25, Information access based on data acquired from real objects). 

1 2. With respect to the claim 50, Miner further teaches said at least one 
network-based information-provider is selected from the group consisting of a 
World-Wide- Web site, intranet site, extranet site, database, knowledge-base, 
search engine, dedicated server and service center (See column 1, lines 25-37, 
The World Wide Web (WWW) is comprised). 

1 3. With respect to the claim 51 , Miner further teaches wherein said 
query being at least one electronic representation of a real-world entity includes 
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an image {See column 1, lines 60-67, a query comprised of an alphanumeric 
search expression or keywords). 

14. With respect to the claim 52, Miner further teaches wherein said 
query being at least one electronic representation of a real-world entity includes a 
sound recording {See column 4, lines 64-67, whereby sounds, such as animal 
sounds, are captured). 

1 5. With respect to the claim 53, Miner further teaches wherein said 
query being at least one electronic representation of a real-world entity includes 
an information segment encoded in electromagnetic radiation {See column 9, 
lines 30-37, an infrared (IR) felectromagnetic radiation j port. Blue Tooth local 
area wireless networking protocol). 

1 6. With respect to the claim 54, Miner further teaches wherein said 
query being at least one electronic representation of a real-word entity selected 
from the group consisting of odor, taste, texture, motion, and vibration {See 
column 8, lines 33-44, a fingerprint (texturej ultrasonic transducer,). 

1 7. With respect to the claim 55, Miner further teaches wherein said 
formulating a query includes fusing data of a plurality of electronic 
representations of real-world entities captured by said data-capture device {See 
column 5, lines 1-6, a user can combine (fusing dataj visual information with 
virtual or invisible information such as GPS. (The examiner would like to state 
that adding GPS data to image data is fusing data according to the specification 
(see page 12, lines 4-11, Introduce data-fusion techniques to incorporate 
additional information to the sensor data ...using GPS technology or using the 
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cellular service provider's information about the device's location, such that a 
geographical component is added to the captured image)j. 

1 8. With respect to the claim 56, Miner further teaches wherein said 
formulating a query includes fusing data inputted by a user with said at least one 
electronic representation of real-world entities captured by said data-capture 
device {See column 5, lines 1-6, a user can combine (fusing dataj visual 
information with virtual or invisible information such as GPS. (The examiner 
would like to state that adding GPS data to image data is fusing data according 
to the specification (see page 12, lines 4-11, Introduce data-fusion techniques to 
incorporate additional information to the sensor data ...using GPS technology or 
using the cellular service provider's information about the device's location, such 
that a geographical component is added to the captured image)| 

1 9. With respect to the claim 57, Miner further teaches further comprising 
presenting information retrieved from said information-provider service by 

way of a user output-device {See column 8, lines 53-63, displays the preliminary 
results to the user by means of an appropriate output device 330). 

20. With respect to the claim 58, Miner further teaches wherein said user 
output device is selected from the group consisting of a visual output device, 
audio output device, textural output device, motion generator, electromagnetic 
transmitter, vibrator and scent generator {See column 8, lines 53-63, displays the 
preliminary results to the user by means of an appropriate output device 330 
such as a liquid crystal display (LCD), an audio message, a speai<er, a monitor,). 
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21 . With respect to the claim 59, IVIiner further teaches comprising 
alerting a relevant party in response to the information retrieved from said 
network-based information-provider according to instructions inputted by a user 
by way of said user- input interface (See column 1 1, lines 17-28, input device 
111 is shown Calertingj acquiring audio signals from an audio search such as a 
bird, to illustrate an audio search capability of the system 10 and see Fig. 10, 
Search Result falertingjj. 

22. With respect to the claim 60, Miner further teaches A portable, 
network-enabled information retrieval device comprising: 

a) a portable, data-capture device for generating electronic 
representations of real-world entities (See column 3, lines 59-67 and see column 
4, lines 1-7, a personal assistant device, and a contextual input device. In use, 
the contextual multimedia association module accesses the Internet (a network- 
enabledj and downloads web documents (retrieve information from a network^ to 
a metadata repository. . . The user captures input data about an object or item of 
interest by means of the contextual input device. ... The personal assistant device 
automatically digitizes (electronic representation j and processes the input data); 

b) a network interface {See column 7, lines 9-20, computers such as 
computers 35, 37, 39, and a variety of other interface devices.); and 

c) a processor, said processor being configured to generate network 
queries for retrieving information from a network containing information-provider, 
said query being at least one electronic representation of a real-world entity 
captured by said data-capture device (See column 8, lines 19-27, an input 
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processor 305 and See column 4, lines 1-7, The personal assistant device 
automatically digitizes (transforming data to an electronic representation j and 
processes the input data, and further automatically formulates a query. . . The 
contextual multimedia association applies the query to numerous data stores, 
optimizes the search results, and then presents the optimized search results to 
the user). 

23. Witli respect to the claim 61 , Miner further teaches wherein said 
data-capture device includes an image capture device so as to be an image- 
based search engine. {See column 1, lines 25-37 Internet search engines to 
retrieve information). 

24. With respect to the claim 62, Miner further teaches wherein said 
data-capture device includes a microphone so as to be an audio-based search 
engine {See column 1 1, lines 17-29, uses a directed microphone as the audio 
sensor 505). 

25. With respect to the claim 63, Miner further teaches wherein said 
data-capture device includes a radio receiver of information segments encoded 
in electromagnetic transmissions (See column 7, lines 20-30, cellular radio 
network 40). 

26. With respect to the claim 64, Miner further teaches wherein said 

data-capture device includes a data-capture device selected from the group 
consisting of scent detector, taste sensor, geophone, motion sensor, acceleration 
meter, wind meter, thermometer, humidly sensor (See column 3, lines 47-57, 
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various sensors such as temperature, pressure, humidity, light level, sound, and 
acceleration sensors). 

27. With respect to tine claim 65, Miner furtlier teaclies wlierein said 
data-capture device is integrated into a cellular phone (See column 7, lines 20- 
30, cellular radio network 40.). 

28. With respect to the claim 66, Miner further teaches wherein said 
data-capture device is integrated into a device selected from the group consisting 
of a wireless phone, netphone, personal digital assistant, portable computer, 
pager, and personal computer (See column 7, lines 20-30, cellular radio network 
40.). 

29. With respect to the claim 67, Miner further teaches wherein said 
query being at least one electronic representation of a real-world entity includes 
an image (See column 1, lines 60-67, a query comprised of an alphanumeric 
search expression or keywords). 

30. With respect to the claim 68, Miner further teaches wherein said 
query being at least one electronic representation of a real-world entity includes a 
sound recording (See column 4, lines 64-67, whereby sounds, such as animal 
sounds, are captured). 

31 . With respect to the claim 69, Miner further teaches wherein said 
query being at least one electronic representation of a real-world entity includes 
an information segment encoded in electromagnetic radiation {See column 9, 
lines 30-37, an infrared (IR) port. Blue Tooth local area wireless networking 
protocol). 
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32. With respect to the claim 70, IVIiner further teaches wherein said 
query being at least one electronic representation of a real-word entity selected 
from the group consisting of odor, taste, texture, motion, and vibration {See 
column 8, lines 33-44, a fingerprint (texture) ultrasonic transducer,). 

33. With respect to the claim 71 , Miner further teaches wherein said 
processor being further configured to fuse data of a plurality of electronic 
representations of real world entities obtained by said data-capture device {See 
column 5, lines 1-6, a user can combine (fusing data) visual information with 
virtual or invisible information sucti as GPS. (The examiner would like to state 
that adding GPS data to image data is fusing data according to the specification 
(see page 12, lines 4-11, Introduce data-fusion techniques to incorporate 
additional information to the sensor data ...using GPS technology or using the 
cellular service provider's information about the device's location, such that a 
geographical component is added to the captured image)! 

34. With respect to the claim 72, Miner further teaches said processor 
being further configured to fuse data inputted by way of a user input-interface 
with data of said electronic representation of real world entities obtained by said 
data-capture device (See column 5, lines 1-6, a user can combine (fusing dataj 
visual information witti virtual or invisible information sucti as GPS. (The 
examiner would like to state that adding GPS data to image data is fusing data 
according to the specification (see page 12, lines 4-1 1 , Introduce data-fusion 
techniques to incorporate additional information to the sensor data ...using GPS 
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technology or using the cellular service provider's information about the device's 
location, such that a geographical component is added to the captured image)! 

35. With respect to the claim 73, Miner further teaches wherein said 
network-based information- provider is implemented as a network-based 
dedicated server configured to perform data-processing on data of the electronic 
representation of real-world entities by said data-capture device (See column 3, 
lines 59-67, installed on a server,.). 

36. With respect to the claim 74, Miner further teaches said at least one 
network-based information-provider is selected from the group consisting of a 
World-Wide- Web site, intranet site, extranet site, database, knowledge-base, 
search engine, dedicated server and service center (See column 1, lines 25-37, 
The World Wide Web (WWW) is comprised). 

37. With respect to the claim 75, Miner further teaches further comprising 
presenting information retrieved from said information-provider service by 

way of a user output-device (See column 8, lines 53-63, displays the preliminary 
results to the user by means of an appropriate output device 330). 

38. With respect to the claim 76, Miner further teaches wherein said user 
output device is selected from the group consisting of a visual output device, 
audio output device, textural output device, motion generator, electromagnetic 
transmitter, vibrator and scent generator (See column 8, lines 53-63, displays the 
preliminary results to the user by means of an appropriate output device 330 
such as a liquid crystal display (LCD), an audio message, a speai<er, a monitor). 
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39. With respect to the claim 77, IVIiner further teaches wherein said 
processor being configurable to alert a relevant party in response to the 
information retrieved from said network-based information-provider according to 
instructions inputted by a user by way of said user- input interface {See column 
1 1, lines 17-28, input device 111 is shown falertingj acquiring audio signals from 
an audio search such as a bird, to illustrate an audio search capability of the 
system 10 and see Fig. 10, Search Result falertingjj. 



Conclusion 

40. Any inquiry concerning this communication or earlier communications 
from the examiner should be directed to FARRUKH HUSSAIN whose telephone 
number is (571)270-5652. The examiner can normally be reached on Monday- 
Thursday, Alt. Friday, 7:30 A.M-5:00 P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, William Vaughn can be reached on 571-272-3922. The 
fax phone number for the organization where this application or proceeding is 
assigned is 571 -273-8300. 
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41 . Information regarding the status of an application may be obtained 
from the Patent Application Information Retrieval (PAIR) system. Status 
information for published applications may be obtained from either Private PAIR 
or Public PAIR. Status information for unpublished applications is available 
through Private PAIR only. For more information about the PAIR system, see 
http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 
(toll-free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
91 99 (IN USA OR CANADA) or 571 -272-1 000. 
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